1.1 Stochastic processes

Let \(\hspace{0.1cm}\mathcal{X}_t\hspace{0.1cm}\) be a random variable (r.v.), for each \(\hspace{0.1cm}t\in T\)

Observation:

\(T\hspace{0.1cm}\) is generally interpreted as moments or periods of time, because one of the most important applications of stochastic processes is time series modeling.

Therefore:

\(X_t\hspace{0.1cm}\) is a random variable ussually used to model the state of a system at time moment \(\hspace{0.06cm}t\hspace{0.06cm}\), or to model a variable of interest at the moment or period \(\hspace{0.06cm}t\).

1.2 Discrete stochastic process

1.3 Continuous stochastic process

1.4 Types of stochastic processes

7.1 Time Series Descomposition in Python

from statsmodels.tsa.seasonal import seasonal_decompose, STL

Weekly_Time_Series_1

advanced_decomposition = STL(Weekly_Time_Series_1.Ventas, period=4).fit()

fig, axs = plt.subplots(nrows=4, ncols=1, sharex=True)

plt.title("Time Series Decomposition",  fontsize = 16)

p1=sns.lineplot(advanced_decomposition.observed, color='red', ax=axs[0])
p2=sns.lineplot(advanced_decomposition.trend, color='red', ax=axs[1])
p3=sns.lineplot(advanced_decomposition.seasonal, color='red', ax=axs[2])
p4=sns.lineplot(advanced_decomposition.resid, color='red', ax=axs[3])

p1.set_ylabel('Observed')
p2.set_ylabel('trend')
p3.set_ylabel('seasonal')
p4.set_ylabel('resid')

p1.set_xticks(np.arange(0 , len(Weekly_Time_Series_1) , 10))

plt.setp(p1.get_xticklabels(), rotation=90)

fig.savefig('p6.jpg', format='jpg', dpi=1200)

plt.show()

The following graph has superimposed the observed series and the trend:

fig, ax = plt.subplots()

p1=sns.lineplot(x="Semana-Mes-Año", y="Ventas", data=Weekly_Time_Series_1 , color='red')
p2=sns.lineplot(advanced_decomposition.trend, color='blue', linestyle='-', label='Trend')

p1.set_xticks(np.arange(0 , len(Weekly_Time_Series_1) , 10))

plt.setp(p1.get_xticklabels(), rotation=90)

plt.title("Trend and Time Series",  fontsize = 17)

fig.savefig('p7.jpg', format='jpg', dpi=1200)

plt.show()

10.1 Strict Stationarity

Given a stochastic process \(\hspace{0.15cm} \mathcal{Y} \hspace{0.1cm}=\hspace{0.1cm} \hspace{0.1cm}\bigl( \hspace{0.06cm} \mathcal{Y}_1 , \mathcal{Y}_2 ,..., \mathcal{Y}_n \hspace{0.06cm}\bigl) \hspace{0.1cm}.\) \(\\[0.25cm]\)

The stochastic process \(\hspace{0.08cm}\mathcal{Y} \hspace{0.08cm}\) is strictly stationary if:

the probability distribution of \(\hspace{0.08cm}\mathcal{Y}_{t}\hspace{0.08cm}\) is the same as that of \(\hspace{0.08cm}\mathcal{Y}_{t+h}\hspace{0.08cm}\), for all \(\hspace{0.1cm}t = 1,2,...,n \rbrace\hspace{0.1cm}\) and \(\hspace{0.1cm} h = 1,2,3,...\\\)

Strict stationarity is a very strong condition, since to prove it we must have the joint distributions for any set of variables in the process. A weaker property, but one which is easier to prove, is weak stationarity.

10.2 Weak Stationarity

Given a stochastic process \(\hspace{0.15cm} \mathcal{Y} \hspace{0.1cm}=\hspace{0.1cm} \hspace{0.1cm}\bigl( \hspace{0.06cm} \mathcal{Y}_1 , \mathcal{Y}_2 ,..., \mathcal{Y}_n \hspace{0.06cm}\bigl) \hspace{0.1cm}.\) \(\\[0.25cm]\)

The stochastic process \(\hspace{0.08cm}\mathcal{Y} \hspace{0.08cm}\) is weakly stationary if: \(\\[0.35cm]\)

• \(\mu_t \hspace{0.08cm}=\hspace{0.08cm} \mu \hspace{0.3cm} , \hspace{0.3cm} \forall\hspace{0.08cm} t = 1,2,...,n \hspace{0.5cm}\text{(weak stationarity in mean)}\\\)

• \(\sigma_t^2 \hspace{0.08cm}=\hspace{0.08cm} \sigma^2 \hspace{0.3cm},\hspace{0.3cm} \forall\hspace{0.08cm} t = 1,2,...,n \hspace{0.5cm}\text{(weak stationarity in variance)}\\\)

• \(\gamma_{t , t - h} \hspace{0.08cm}=\hspace{0.08cm} \gamma_h \hspace{0.4cm},\hspace{0.4cm} \gamma_{h}\hspace{0.08cm}=\hspace{0.08cm} \gamma_{-h} \hspace{0.4cm},\hspace{0.4cm} \forall\hspace{0.08cm} h = 0 , \pm 1 , \pm 2 ,... \hspace{0.5cm}\text{(weak stationarity in covariance)}\\\)

  Where: \(\hspace{0.2cm}\gamma_{t , t - h} \hspace{0.08cm}=\hspace{0.08cm}Cov(\mathcal{Y}_t,\mathcal{Y}_{t-h}) \hspace{0.08cm}=\hspace{0.08cm}E[(\mathcal{Y}_t - \mu)\cdot (\mathcal{Y}_{t-h} - \mu)]\\\)

The first two conditions indicate that the mean and variance are constant.

The third indicates that the covariance between two variables depends only on their temporal separation.

In a stationary process the autocovariances and autocorrelations depend only on the lag between the variables and, in particular, the relationship between \(\hspace{0.08cm}\mathcal{Y}_t\hspace{0.08cm}\) and \(\hspace{0.08cm}\mathcal{Y}_{t+h}\hspace{0.08cm}\) , is always equal to the relationship between \(\hspace{0.08cm}\mathcal{Y}_t\hspace{0.08cm}\) and \(\hspace{0.08cm}\mathcal{Y}_{t-h}\hspace{0.08cm}\), this can be summarize as \(\hspace{0.08cm}\gamm_{t,t-h} \hspace{0.08cm}=\hspace{0.08cm} \gamma_{h}\hspace{0.08cm}=\hspace{0.08cm} \gamma_{-h} \hspace{0.08cm}=\hspace{0.08cm} \gamma_{t,t+h}\hspace{0.08cm}\) .

As a result, in a weak stationary processes:

\[\rho_{t, t-h} \hspace{0.08cm}=\hspace{0.08cm} \dfrac{\gamma_{t , t - h}}{\sqrt{\sigma_t^2 \cdot \sigma_{t-h}^2}} \hspace{0.08cm}=\hspace{0.08cm} \dfrac{\gamma_h}{\sqrt{\sigma^2 \cdot \sigma^2}} \hspace{0.08cm}=\hspace{0.08cm} \dfrac{\gamma_h}{\sigma^2}\hspace{0.08cm} =\hspace{0.08cm} \dfrac{\gamma_h}{\gamma_0} \hspace{0.08cm}=\hspace{0.08cm} \rho_h\]

And also it’s fulfilled that \(\hspace{0.08cm}\rho_h\hspace{0.08cm} = \hspace{0.08cm}\rho_{-h}\hspace{0.08cm}\).

Observation:

Weak stationarity doesn’t guarantee full process stability, namely, the probability distribution of \(\hspace{0.08cm}\mathcal{Y}_t\hspace{0.08cm}\) may change for different \(\hspace{0.08cm}t\hspace{0.08cm}\) values.

But, if it is assume that \(\hspace{0.08cm}\mathcal{Y}_1,\dots , \mathcal{Y}_n \hspace{0.08cm}\sim\hspace{0.08cm} NM_n(\mathbf{\mu}, \mathbf{\Sigma})\hspace{0.08cm}\), then, weak stationariy is equivalent to strict stationarity.

10.3 Asymptotic Weak Stationarity

Given a stochastic process \(\hspace{0.15cm} \mathcal{Y} \hspace{0.1cm}=\hspace{0.1cm} \hspace{0.1cm}\bigl( \hspace{0.06cm} \mathcal{Y}_1 , \mathcal{Y}_2 ,..., \mathcal{Y}_n \hspace{0.06cm}\bigl) \hspace{0.1cm}.\) \(\\[0.25cm]\)

The stochastic process \(\hspace{0.08cm}\mathcal{Y} \hspace{0.08cm}\) is asymptotically stationary in a weak sense if: \(\\[0.35cm]\)

• \(\mu_t \hspace{0.08cm}=\hspace{0.08cm} \mu \hspace{0.3cm} , \hspace{0.3cm} \forall\hspace{0.08cm} t \rightarrow \infty \hspace{0.5cm}\text{(asymptotic weak stationarity in mean)}\\\)

• \(\sigma_t^2 \hspace{0.08cm}=\hspace{0.08cm} \sigma^2 \hspace{0.3cm},\hspace{0.3cm} \forall\hspace{0.08cm} t \rightarrow \infty \hspace{0.5cm}\text{(asymptotic weak stationarity in variance)}\\\)

• \(\gamma_{t , t - h} \hspace{0.08cm}=\hspace{0.08cm} \gamma_h \hspace{0.4cm},\hspace{0.4cm} \gamma_{h}\hspace{0.08cm}=\hspace{0.08cm} \gamma_{-h} \hspace{0.4cm},\hspace{0.4cm} \forall\hspace{0.08cm} h = 0 , \pm 1 , \pm 2 ,... \hspace{0.5cm}\text{(weak stationarity in covariance)}\\\)

10.4 White Noise Process

Given a stochastic process \(\hspace{0.15cm} \mathcal{Y} \hspace{0.08cm}=\hspace{0.08cm} \hspace{0.1cm}\bigl( \hspace{0.06cm} \mathcal{Y}_1 , \mathcal{Y}_2 ,..., \mathcal{Y}_n \hspace{0.06cm}\bigl) \hspace{0.1cm}.\) \(\\[0.25cm]\)

The process \(\hspace{0.08cm}\mathcal{Y}\hspace{0.08cm}\) is a white noise process if: \(\\[0.35cm]\)

• \(E[\mathcal{Y}_t] \hspace{0.08cm}=\hspace{0.08cm} 0 \hspace{0.3cm}, \hspace{0.3cm}\forall\hspace{0.08cm} t=1,\dots , n\\\)

• \(Var(\mathcal{Y}_t)=\sigma^2 \hspace{0.3cm}, \hspace{0.3cm}\forall\hspace{0.08cm} t=1,\dots , n\\\)

• \(\gamma_{t,t+h} \hspace{0.08cm}= \hspace{0.08cm}Cov(\mathcal{Y}_t, \mathcal{Y}_{t+h})\hspace{0.08cm}=\hspace{0.08cm}0 \hspace{0.3cm}, \hspace{0.3cm}\forall\hspace{0.08cm} t=1,\dots , n\\\)

Observation:

A white noise process is a weak stationarity process.

11.1 Expression of AR(1) as a function of the lag operator

We can express \(\hspace{0.08cm}AR(2)\hspace{0.08cm}\) process using the lag operator as follows: \(\\[0.3cm]\)

\[\left(1 - \phi_1 \cdot B\right) \cdot \mathcal{Y}_t \hspace{0.08cm} =\hspace{0.08cm} \phi_0 \hspace{0.08cm}+\hspace{0.08cm} \varepsilon_t\]

Where:

\(B\hspace{0.08cm}\) is the lag operator and is defined as follows:

\[B\cdot \mathcal{Y}_t \hspace{0.08cm}=\hspace{0.08cm} \mathcal{Y}_{t-1}\]

\[B^h\cdot \mathcal{Y}_t \hspace{0.08cm}=\hspace{0.08cm} \mathcal{Y}_{t-h} \hspace{0.3cm} , \hspace{0.3cm} h=1,2,\dots\]

11.2 Mean of AR(1) process

In general, for an \(\hspace{0.08cm}AR(1)\hspace{0.08cm}\) process we have the following:

\[\mu_t \hspace{0.08cm}=\hspace{0.08cm} E[\mathcal{Y}_t] \hspace{0.08cm}=\hspace{0.08cm} \phi_0 \cdot \sum_{i=0}^{t-2}\hspace{0.08cm} \phi_1^i \hspace{0.08cm}+\hspace{0.08cm} \phi_1^{t-1}\cdot \mu_1\]

Where: \(\hspace{0.12cm}\mu_1 \hspace{0.08cm}=\hspace{0.08cm} E[\mathcal{Y}_1]\\\)

As we can see, \(\hspace{0.08cm}\mu_t\hspace{0.08cm}\) depends on \(\hspace{0.08cm}t\hspace{0.08cm}\) so, It isn’t necessarily constant. That means \(\hspace{0.08cm}AR(1)\hspace{0.08cm}\) isn’t necessarily a weak stationary process. \(\\[1.5cm]\)

• If \(\hspace{0.1cm}| \phi_1 | < 1\hspace{0.1cm}\) , then:

  \[\mu_t \hspace{0.08cm}=\hspace{0.08cm} \mu_{\mathcal{Y}} \hspace{0.08cm}=\hspace{0.08cm} \dfrac{\phi_0}{1-\phi_1} \hspace{0.35cm} ,\hspace{0.35cm} t \rightarrow \infty \\\]

  So, the \(\hspace{0.08cm}AR(1)\hspace{0.08cm}\) process is weakly stationary in mean, in an asymptotic sense, because the mean converge to a constant value when \(\hspace{0.08cm}t\hspace{0.08cm}\) is large enough. \(\\[0.3cm]\)

  Using that fact, we will consider the following approximation:\(\\[0.3cm]\)

  \[\mu_t\hspace{0.08cm} \approx\hspace{0.08cm} \mu_{\mathcal{Y}} \hspace{0.08cm}=\hspace{0.08cm} \dfrac{\phi_0}{1-\phi_1} \hspace{0.3cm},\hspace{0.3cm} \forall\hspace{0.08cm} t = 1,2,\dots , n\] \(\\[0.7cm]\)

• If \(\hspace{0.1cm}| \phi_1 | \geq 1\hspace{0.1cm}\) , then: \(\\[0.4cm]\)

  \[\mu_t\hspace{0.2cm} \text{doesn't converge to a constant value.}\hspace{0.3cm},\hspace{0.3cm} \forall\hspace{0.08cm} t = 1,2,\dots , n \\\]

  So, the \(\hspace{0.08cm}AR(1)\hspace{0.08cm}\) process isn’t weak stationary in mean, and by extension, It isn’t weak stationary, in general, so neither in a strict sense.

Proof:

Using the definition of the \(\hspace{0.08cm}AR(1)\hspace{0.08cm}\) process, by recursion, we can proof the following statement:

\[\mathcal{Y}_t \hspace{0.08cm}=\hspace{0.08cm} \phi_0 \cdot \sum_{i=0}^{t-2}\hspace{0.08cm} \phi_1^i \hspace{0.08cm}+\hspace{0.08cm} \phi_1^{t-1}\cdot \mathcal{Y}_1 \hspace{0.08cm}+\hspace{0.08cm} \sum_{i=0}^{t-2} \hspace{0.08cm}\phi_1^i \cdot \varepsilon_{t-i} \hspace{0.3cm},\hspace{0.3cm} t=2,3,\dots,n \\\]

So,taking the expectation we have:

\[\mu_t = E[\mathcal{Y}_t] \hspace{0.08cm}=\hspace{0.08cm} \phi_0 \cdot \sum_{i=0}^{t-2}\hspace{0.08cm} \phi_1^i \hspace{0.08cm}+\hspace{0.08cm} \phi_1^{t-1}\cdot \mu_1 \hspace{0.3cm},\hspace{0.3cm} t=2,3,\dots,n\]

Therefore, \(\hspace{0.08cm}\mu_t\hspace{0.08cm}\) depends on \(\hspace{0.08cm}t\hspace{0.08cm}\) and isn’t necessarily constant. \(\\[1cm]\)

• If \(\hspace{0.1cm}| \phi_1 | < 1\hspace{0.1cm}\) , then:

  \[\sum_{i=0}^{t-2}\hspace{0.08cm} \phi_1^i \hspace{0.1cm}=\hspace{0.1cm} \dfrac{\hspace{0.08cm}1-\phi_1^{t-1}\hspace{0.08cm}}{1-\phi_1}\\\]

  So, we have,:

  \[\mu_t \hspace{0.08cm}=\hspace{0.08cm} \phi_0 \cdot\dfrac{1-\phi_1^{t-1}}{1-\phi_1} \hspace{0.08cm}+\hspace{0.08cm} \phi_1^{t-1}\cdot \mu_1\\\]

  And, under the assumption of \(\hspace{0.1cm}| \phi_1 | < 1\hspace{0.1cm}\), we also have:

  \[\underset{t \rightarrow \infty}{\text{lim}} \hspace{0.08cm} \phi_1^{t-1} \hspace{0.08cm}=\hspace{0.08cm} 0\\\]

  Therefore:

  \[\underset{t \rightarrow \infty}{\text{lim}} \hspace{0.08cm} \mu_t \hspace{0.08cm}=\hspace{0.08cm} \phi_0 \cdot\dfrac{1}{1-\phi_1}\\\]

  So, the mean \(\hspace{0.08cm}\mu_t\hspace{0.08cm}\) converge to a constant value when \(t\) is large. So, the process \(AR(1)\) process is weakly stationary in a asymptotic sense. \(\\[1.5cm]\)

• If \(\hspace{0.1cm}| \phi_1 | \geq 1\hspace{0.1cm}\) , then: \(\\[0.7cm]\)

  The sum \(\hspace{0.12cm}\sum_{i=0}^{t-2}\hspace{0.08cm} \phi_1^i\hspace{0.1cm}\) doesn’t converge to a constant value.

  And the value of \(\hspace{0.1cm}\phi_1^{t-1}\hspace{0.1cm}\) depends on \(\hspace{0.08cm}t\hspace{0.08cm}\) and doen’t converge to \(\hspace{0.08cm}0\hspace{0.1cm}\) when \(\hspace{0.08cm}t\rightarrow \infty\)

  So, the mean \(\hspace{0.08cm}\mu_t\hspace{0.08cm}\) doesn’t converge to a constant value. Then, the \(\hspace{0.08cm}AR(1)\hspace{0.08cm}\) process is not stationary, neither in the weak nor in the strict sense.

11.3 Variance of AR(1) process

In general, for an \(\hspace{0.08cm}AR(1)\hspace{0.08cm}\) process we have the following:

\[\sigma_t^2 \hspace{0.08cm}=\hspace{0.08cm} Var(\mathcal{Y}_t) \hspace{0.08cm}=\hspace{0.08cm} \phi_1^{2(t-1)} \cdot \sigma_1^2 \hspace{0.08cm}+\hspace{0.08cm} Var\left(\hspace{0.08cm} \sum_{i=0}^{t-2} \phi_1^i \cdot \varepsilon_{t-i} \hspace{0.08cm} \right)\\\]

As we can see, \(\hspace{0.1cm}\sigma_t^2\hspace{0.1cm}\) depends on \(\hspace{0.08cm}t\hspace{0.08cm}\) so, not is necessarily constant. That means \(\hspace{0.08cm}AR(1)\hspace{0.08cm}\) isn’t necessarily a weak stationary process. \(\\[1.5cm]\)

• If \(\hspace{0.1cm}| \phi_1 | < 1\hspace{0.1cm}\) , then:

  \[\sigma_t^2 \hspace{0.08cm}=\hspace{0.08cm} \sigma_{\mathcal{Y}}^2 \hspace{0.08cm}=\hspace{0.08cm} \dfrac{\sigma^2}{1-\phi_1^2} \hspace{0.35cm} ,\hspace{0.35cm} t \rightarrow \infty\\\]

  So, the AR(1) process is weakly stationary in variance, in an asymptotic sense, because the variance converge to a constant value when \(\hspace{0.08cm}t\hspace{0.08cm}\) is large enough. \(\\[0.3cm]\)

  Using that fact, we will consider the following approximation:\(\\[0.3cm]\)

  \[\sigma_t^2\hspace{0.08cm} \approx\hspace{0.08cm} \sigma_{\mathcal{Y}}^2 \hspace{0.08cm}=\hspace{0.08cm} \dfrac{\sigma^2}{1-\phi_1^2} \hspace{0.3cm},\hspace{0.3cm} \forall\hspace{0.08cm} t = 1,2,\dots , n \] \(\\[0.7cm]\)

• If \(\hspace{0.1cm}| \phi_1 | \geq 1\hspace{0.1cm}\) , then: \(\\[0.35cm]\)

  \[\hspace{0.08cm}\sigma_t^2\hspace{0.13cm} \text{doesn't converge to a constant value.} \hspace{0.3cm},\hspace{0.3cm} \forall\hspace{0.08cm} t = 1,2,\dots , n\\\]

  So, the process \(\hspace{0.08cm}AR(1)\hspace{0.08cm}\) isn’t weak stationary in variance, and by extension, the process \(\hspace{0.08cm}AR(1)\hspace{0.08cm}\) isn’t a weak stationary process, so neither in a strict sense. \(\\[1cm]\)

Proof:

Using the definition of the \(\hspace{0.08cm}AR(1)\hspace{0.08cm}\) process by recursion we can proof the following statement:

\[\mathcal{Y}_t \hspace{0.08cm}=\hspace{0.08cm} \phi_0 \cdot \sum_{i=0}^{t-2}\hspace{0.08cm} \phi_1^i \hspace{0.08cm}+\hspace{0.08cm} \phi_1^{t-1}\cdot \mathcal{Y}_1 \hspace{0.08cm}+\hspace{0.08cm} \sum_{i=0}^{t-2} \hspace{0.08cm}\phi_1^i \cdot \varepsilon_{t-i} \hspace{0.3cm},\hspace{0.3cm} t=2,3,\dots,n\]

So, taking the variance we have:

\[\sigma_t^2 \hspace{0.08cm}=\hspace{0.08cm} Var(\mathcal{Y}_t) \hspace{0.08cm}=\hspace{0.08cm} \phi_1^{2(t-1)} \cdot \sigma_1^2 \hspace{0.08cm}+\hspace{0.08cm} Var\left(\sum_{i=0}^{t-2} \phi_1^i \cdot \varepsilon_{t-i}\right)\]

We have that:

\[Var\left(\sum_{i=0}^{t-2} \phi_1^i \cdot \varepsilon_{t-i}\right) \hspace{0.08cm}=\hspace{0.08cm} Var(\varepsilon_t + \phi_1\cdot\varepsilon_{t-1} + \phi_1^2\cdot \varepsilon_{t-2} +\dots + \phi_1^{t-2}\varepsilon_2)\hspace{0.08cm}= \\[1cm] =\hspace{0.08cm} \sigma^2 + (\phi_1)^2 \cdot \sigma^2 + (\phi_1^2)^2 \cdot \sigma^2 + \dots + (\phi_1^{t-2})^2 \cdot \sigma^2 \hspace{0.08cm}+\hspace{0.08cm} 2\cdot \sum_{i\neq j \\ i,j=2,...,t-2} Cov(\varepsilon_i , \varepsilon_j) \cdot a_i \cdot a_j\]

Where: \(\hspace{0.15cm}a_i = \phi_1^{t-i} \hspace{0.25cm},\hspace{0.25cm} i=2,3,\dots ,n\)

Taking into account that \(\hspace{0.08cm}Cov(\varepsilon_i , \varepsilon_j)=0 \hspace{0.15cm},\hspace{0.15cm} \forall i,j\hspace{0.12cm}\), then: \(\\[0.5cm]\)

\[Var\left(\sum_{i=0}^{t-2} \phi_1^i \cdot \varepsilon_{t-i}\right) \hspace{0.08cm}=\hspace{0.08cm} \sigma^2 + (\phi_1)^2 \cdot \sigma^2 + (\phi_1^2)^2 \cdot \sigma^2 + \dots + (\phi_1^{t-2})^2 \cdot \sigma^2 \hspace{0.08cm} = \\[0.5cm] = \hspace{0.08cm} \sum_{i=0}^{t-2} (\phi_1^i)^2 \cdot \sigma^2 = \sum_{i=0}^{t-2} \phi_1^{2i} \cdot \sigma^2\]

Therefore:

\[\sigma_t^2 \hspace{0.08cm}=\hspace{0.08cm} Var(\mathcal{Y}_t) \hspace{0.08cm}=\hspace{0.08cm} \phi_1^{2(t-1)} \cdot \sigma_1^2 \hspace{0.08cm}+\hspace{0.08cm} \sigma^2 \cdot \sum_{i=0}^{t-2} \phi_1^{2i}\]

As we can see, \(\hspace{0.08cm}\sigma_t^2\hspace{0.08cm}\) depend on \(\hspace{0.08cm}t\hspace{0.08cm}\), so it is not necessarily the same constant for each \(\hspace{0.08cm}t\hspace{0.08cm}.\\\)

• If \(\hspace{0.1cm}| \phi_1 | < 1\hspace{0.1cm}\) , then:

  \[\sum_{i=0}^{t-2}\hspace{0.08cm} (\phi_1^2)^i \hspace{0.1cm}=\hspace{0.1cm} \dfrac{\hspace{0.08cm}1-(\phi_1^2)^{t-1}\hspace{0.08cm}}{1-\phi_1^2}\\\]

  So, we have:

  \[\sigma_t^2 \hspace{0.08cm}=\hspace{0.08cm} Var(\mathcal{Y}_t) \hspace{0.08cm}=\hspace{0.08cm} \phi_1^{2(t-1)} \cdot \sigma_1^2 \hspace{0.08cm}+\hspace{0.08cm} \dfrac{\hspace{0.08cm}1-(\phi_1^2)^{t-1}\hspace{0.08cm}}{1-\phi_1^2} \cdot \sigma^2\]

And we also have:

\[\underset{t\rightarrow \infty}{\text{lim}} \hspace{0.08cm} \phi_1^{2(t-1)} = 0\]

Therefore:

\[\underset{t\rightarrow \infty}{\text{lim}} \hspace{0.08cm} \sigma_t^2 \hspace{0.1cm}=\hspace{0.1cm} \dfrac{\hspace{0.08cm}1 \hspace{0.08cm}}{1-\phi_1^2} \cdot \sigma^2\hspace{0.1cm}=\hspace{0.1cm} \dfrac{\hspace{0.08cm}\sigma^2 \hspace{0.08cm}}{1-\phi_1^2}\\\]

• If \(\hspace{0.1cm}| \phi_1 | \geq 1\hspace{0.1cm}\), then: \(\\[0.35cm]\)

  The sum \(\hspace{0.12cm}\sum_{i=0}^{t-2}\hspace{0.08cm} (\phi_1^2)^i\hspace{0.1cm}\) doesn’t converge to a constant value, is a divergent sum. \(\\[0.3cm]\)

  And the value of \(\hspace{0.1cm}\phi_1^{2(t-1)}\hspace{0.1cm}\) depends on \(\hspace{0.08cm}t\hspace{0.08cm}\) and doen’t converge to \(\hspace{0.08cm}0\hspace{0.1cm}\) when \(\hspace{0.08cm}t\rightarrow \infty\). \(\\[0.3cm]\)

  So, the variance \(\hspace{0.08cm}\sigma^2_t\hspace{0.08cm}\) doesn’t converge to a constant value. Then, the \(\hspace{0.08cm}AR(1)\hspace{0.08cm}\) process is not stationary, neither in the weak nor in the strict sense.

11.4 Autocovariance of AR(1) process

If \(\hspace{0.08cm}AR(1)\hspace{0.08cm}\) process isn’t weakly stationary in autocovariance, then isn’t a close expression for the covariance \(\hspace{0.08cm}Cov(\mathcal{Y}_t , \mathcal{Y}_{t-h})\hspace{0.08cm}\).

Assuming that \(\hspace{0.08cm}AR(1)\hspace{0.08cm}\) process is weakly stationary in covariance, we have the following: \(\\[0.3cm]\)

\[\gamma_{h} \hspace{0.1cm}=\hspace{0.1cm} Cov(\mathcal{Y}_t , \mathcal{Y}_{t-h}) \hspace{0.1cm}=\hspace{0.1cm} \dfrac{\phi_1^{|h|} }{1-\phi_1^2}\cdot \sigma^2\hspace{0.1cm}=\hspace{0.1cm} \phi_1^{|h|} \cdot \sigma_{\mathcal{Y}}^2 \hspace{0.4cm},\hspace{0.4cm} \forall\hspace{0.08cm} t=1,2,\dots ,n \hspace{0.2cm},\hspace{0.2cm} \forall\hspace{0.08cm} h=0,\pm 1, \pm 2,\dots \\\]

Note that: \(\hspace{0.1cm}\gamma_{h} = \gamma_{-h}\)

And also we have the following relation between \(\hspace{0.08cm}\gamma_{h}\hspace{0.08cm}\) and \(\hspace{0.08cm}\gamma_{h-1}\hspace{0.08cm}\) : \(\\[0.3cm]\)

\[\gamma_{h} \hspace{0.1cm}=\hspace{0.1cm} \phi_1 \cdot \gamma_{h-1} \hspace{0.4cm},\hspace{0.4cm} \forall\hspace{0.08cm} h=0,\pm 1, \pm 2,\dots\]

Observation:

If \(\hspace{0.08cm}| \phi_1 | < 1\hspace{0.08cm}\) , then \(\hspace{0.08cm}\gamma_{h}\hspace{0.08cm}\) decrease when \(\hspace{0.08cm}h\hspace{0.08cm}\) increases. This means that the relation between two variables of the process decreases when the temporal lag between them increases.

Proof:

Using the definition of an \(\hspace{0.08cm}AR(1)\hspace{0.08cm}\) process and some covariance properties, we obtain the following: \(\\[0.35cm]\)

\[\gamma_{h} \hspace{0.1cm}=\hspace{0.1cm}Cov(\mathcal{Y}_t , \mathcal{Y}_{t-h}) \hspace{0.08cm}=\hspace{0.08cm} Cov(\phi_0 + \phi_1\cdot \mathcal{Y}_{t-1} + \varepsilon_t \hspace{0.08cm},\hspace{0.08cm} \mathcal{Y}_{t-h}) \hspace{0.08cm}=\hspace{0.08cm} Cov(\phi_0 \hspace{0.08cm},\hspace{0.08cm} \mathcal{Y}_{t-h}) \hspace{0.08cm}+\hspace{0.08cm} Cov(\phi_1\cdot \mathcal{Y}_{t-1} \hspace{0.08cm},\hspace{0.08cm} \mathcal{Y}_{t-h}) \hspace{0.08cm}+\hspace{0.08cm} Cov(\varepsilon_t \hspace{0.08cm},\hspace{0.08cm} \mathcal{Y}_{t-h}) \hspace{0.08cm}= \\[0.55cm] =\hspace{0.08cm} \phi_1\cdot Cov( \mathcal{Y}_{t-1} \hspace{0.08cm},\hspace{0.08cm} \mathcal{Y}_{t-h})\hspace{0.08cm}=\hspace{0.08cm} \phi_1 \cdot Cov( \mathcal{Y}_{t-1} \hspace{0.08cm},\hspace{0.08cm} \mathcal{Y}_{t-1-(h-1)}) \hspace{0.08cm}=\hspace{0.08cm} \phi_1 \cdot \gamma_{h-1}\\\]

Where:

• \(Cov(\phi_0 \hspace{0.08cm},\hspace{0.08cm} \mathcal{Y}_{t-h}) \hspace{0.08cm}=\hspace{0.08cm} 0\hspace{0.08cm}\) because \(\hspace{0.08cm}\phi_0\hspace{0.08cm}\) is a constant. \(\\[0.5cm]\)

• \(Cov(\varepsilon_t \hspace{0.08cm},\hspace{0.08cm} \mathcal{Y}_{t-h}) \hspace{0.08cm}=\hspace{0.08cm} 0\hspace{0.08cm}\) by initial assumption of \(\hspace{0.08cm}AR(1)\hspace{0.08cm}\) process. \(\\[0.5cm]\)

• \(Cov( \mathcal{Y}_{t-1} \hspace{0.08cm},\hspace{0.08cm} \mathcal{Y}_{t-h})\hspace{0.08cm}=\hspace{0.08cm}Cov( \mathcal{Y}_{t-1} \hspace{0.08cm},\hspace{0.08cm} \mathcal{Y}_{t-1-(h-1)}) \hspace{0.08cm}=\hspace{0.08cm} \gamma_{h-1}\hspace{0.08cm}\) because it has assumed that AR(1) is weakly stationary in covariance, so that, the covariance between two random variables of the process only depend on the time-lag between them. \(\\[0.35cm]\)

Now we have to proof: \(\hspace{0.13cm}\gamma_{h} \hspace{0.1cm}=\hspace{0.1cm} Cov(\mathcal{Y}_t , \mathcal{Y}_{t-h}) \hspace{0.1cm}=\hspace{0.1cm} \dfrac{\phi_1^k }{1-\phi_1^2}\cdot \sigma^2\hspace{0.1cm}.\\\)

\[\hspace{0.1cm}\gamma_{1} \hspace{0.1cm}=\hspace{0.1cm} Cov(\mathcal{Y}_t , \mathcal{Y}_{t-1}) \hspace{0.1cm}=\hspace{0.1cm} Cov(\phi_0+\phi_1\cdot \mathcal{Y}_{t-1} + \varepsilon_t , \mathcal{Y}_{t-1} \hspace{0.1cm}=\hspace{0.1cm} Cov(\phi_0 , \mathcal{Y}_{t-1}) + \phi_1\cdot Cov(\mathcal{Y}_{t-1} , \mathcal{Y}_{t-1} ) + Cov(\varepsilon_t , \mathcal{Y}_{t-1})\hspace{0.1cm}= \\[0.4cm] =\hspace{0.1cm} \phi_1 \cdot \sigma_{t-1}^2 \hspace{0.1cm}=\hspace{0.1cm} \phi_1 \cdot \sigma_{\mathcal{Y}}^2 \hspace{0.1cm}=\hspace{0.1cm} \phi_1 \cdot \dfrac{\sigma^2}{1-\phi_1^2}\]

Now, using the fact \(\hspace{0.08cm} \gamma_{h} \hspace{0.08cm}=\hspace{0.08cm} \phi_1 \cdot \gamma_{h-1}\hspace{0.08cm}\) in a recursive way, we have:

\[\gamma_{2} \hspace{0.08cm}=\hspace{0.08cm} \phi_1 \cdot \gamma_{1} \hspace{0.08cm}=\hspace{0.08cm} \phi_1 \cdot \phi_1 \cdot \dfrac{\sigma^2}{1-\phi_1^2} \hspace{0.08cm}=\hspace{0.08cm}\phi_1^2 \cdot \dfrac{\sigma^2}{1-\phi_1^2}\]

\[\gamma_{3} \hspace{0.08cm}=\hspace{0.08cm} \phi_1 \cdot \gamma_{2} \hspace{0.08cm}=\hspace{0.08cm} \phi_1 \cdot \phi_1^2 \cdot \dfrac{\sigma^2}{1-\phi_1^2} \hspace{0.08cm}=\hspace{0.08cm}\phi_1^3 \cdot \dfrac{\sigma^2}{1-\phi_1^2}\]

\[\gamma_{4} \hspace{0.08cm}=\hspace{0.08cm} \phi_1 \cdot \gamma_{3} \hspace{0.08cm}=\hspace{0.08cm} \phi_1 \cdot \phi_1^3 \cdot \dfrac{\sigma^2}{1-\phi_1^2} \hspace{0.08cm}=\hspace{0.08cm}\phi_1^4 \cdot \dfrac{\sigma^2}{1-\phi_1^2}\]

In general:

\[\gamma_{h} \hspace{0.08cm}=\hspace{0.08cm} \phi_1 \cdot \gamma_{h-1} \hspace{0.08cm}=\hspace{0.08cm} \phi_1 \cdot \phi_h^{h-1} \cdot \dfrac{\sigma^2}{1-\phi_1^2} \hspace{0.08cm}=\hspace{0.08cm}\phi_1^h \cdot \dfrac{\sigma^2}{1-\phi_1^2}\]

11.5 Autocorrelation of AR(1) process

As we have said, \(\hspace{0.08cm}AR(1)\hspace{0.08cm}\) process isn’t weakly stationary in covariance, then isn’t a close expression for the covariance \(\hspace{0.08cm}Cov(\mathcal{Y}_t , \mathcal{Y}_{t-h})\hspace{0.08cm}\), so, by extension, neither for the correlation, since it depend on covariance.

Assuming that \(\hspace{0.08cm}AR(1)\hspace{0.08cm}\) process is weakly stationary in covariance, using the relation between correlation and covariance, we have the following: \(\\[0.4cm]\)

\[\rho_h \hspace{0.08cm}=\hspace{0.08cm} Corr(\mathcal{Y}_t , \mathcal{Y}_{t-h}) \hspace{0.08cm}=\hspace{0.08cm} \dfrac{Cov(\mathcal{Y}_t , \mathcal{Y}_{t-h})}{\sqrt{Var(\mathcal{Y}_t)\cdot Var(\mathcal{Y}_{t-h})}} \hspace{0.08cm}=\hspace{0.08cm}\dfrac{\hspace{0.08cm}\gamma_{h}\hspace{0.08cm}}{\sigma^2_{\mathcal{Y}}}\]

Note that: \(\hspace{0.11cm}\rho_h = \rho_{-h}\)

11.6 Weak Stationarity Condition of an AR(1)

We will summarize the above points. \(\\[0.3cm]\)

• In a \(\hspace{0.08cm}AR(1)\hspace{0.08cm}\) process, if \(\hspace{0.08cm}| \phi_1 | < 1\hspace{0.08cm}\) , we have the following: \(\\[0.4cm]\)

  \[\mu_t\hspace{0.08cm} \approx\hspace{0.08cm} \mu_{\mathcal{Y}} \hspace{0.08cm}=\hspace{0.08cm} \dfrac{\phi_0}{1-\phi_1} \hspace{0.3cm},\hspace{0.3cm} \forall\hspace{0.08cm} t = 1,2,\dots , n\\\]

  \[\sigma_t^2\hspace{0.08cm} \approx\hspace{0.08cm} \sigma_{\mathcal{Y}}^2 \hspace{0.08cm}=\hspace{0.08cm} \dfrac{\sigma^2}{1-\phi_1^2} \hspace{0.3cm},\hspace{0.3cm} \forall\hspace{0.08cm} t = 1,2,\dots , n \\[0.4cm]\]

  So, if \(\hspace{0.1cm}| \phi_1 | < 1\hspace{0.1cm}\), then \(\hspace{0.08cm}AR(1)\hspace{0.08cm}\) is approximately weakly stationary in mean and variance, or being more precise, is asymptotically weakly stationary in mean and variance. \(\\[0.35cm]\)

  For this reason, \(\hspace{0.08cm}| \phi_1 | < 1\hspace{0.08cm}\) is known as weak stationarity condition, for mean and variance, of an \(\hspace{0.08cm}AR(1)\hspace{0.08cm}\) . \(\\[1.5cm]\)

• And, if we assume that \(\hspace{0.08cm}AR(1)\hspace{0.08cm}\) process is weakly stationary in covariance, we have:

  \[\gamma_{h} \hspace{0.1cm}=\hspace{0.1cm} Cov(\mathcal{Y}_t , \mathcal{Y}_{t-h}) \hspace{0.1cm}=\hspace{0.1cm} \dfrac{\phi_1^h }{1-\phi_1^2}\cdot \sigma^2 \hspace{0.08cm}=\hspace{0.08cm} \phi_1^h \cdot \sigma_{\mathcal{Y}}^2 \hspace{0.4cm},\hspace{0.4cm} \forall\hspace{0.08cm} t=1,2,\dots ,n \hspace{0.2cm},\hspace{0.2cm} \forall\hspace{0.08cm} h=0,\pm 1, \pm 2,\dots \\\]

  And using correlation definition we also have:

  \[\rho_h \hspace{0.08cm}=\hspace{0.08cm} Corr(\mathcal{Y}_t , \mathcal{Y}_{t-h}) \hspace{0.08cm}=\hspace{0.08cm} \dfrac{\hspace{0.08cm}\gamma_{h}\hspace{0.08cm}}{\sigma^2_{\mathcal{Y}}}\]

If \(\hspace{0.1cm}| \phi_1 | < 1\hspace{0.1cm}\) , then \(\hspace{0.08cm}\gamma_{h}\hspace{0.08cm}\) decrease when \(\hspace{0.08cm}h\hspace{0.08cm}\) increases. That means that the relation between two variables of the process decreases when the temporal lag between them increases. In other words, \(\hspace{0.08cm}\gamma_{1} \hspace{0.08cm}>\hspace{0.08cm} \gamma_{2} \hspace{0.08cm}>\hspace{0.08cm} \gamma_{3} \hspace{0.08cm}>\hspace{0.08cm} \dots \hspace{0.12cm}\) And, by extension \(\hspace{0.08cm}\rho_{1} \hspace{0.08cm}>\hspace{0.08cm} \rho_{2} \hspace{0.08cm}>\hspace{0.08cm} \rho_{3} \hspace{0.08cm}>\hspace{0.08cm} \dots \hspace{0.08cm}\)

12.1 Estimating past values of the process

Using an \(\hspace{0.08cm}AR(1)\hspace{0.08cm}\) model, the estimation of the past (observed) values \(\hspace{0.1cm}y_t \hspace{0.1cm},\hspace{0.1cm} t=1,...,n\) of the process is done using the following equation: \(\\[0.3cm]\)

\[\widehat{y}_t \hspace{0.1cm}=\hspace{0.1cm} \widehat{\phi}_0 \hspace{0.1cm}+\hspace{0.1cm} \widehat{\phi}_1 \cdot y_{t-1} \hspace{0.35cm},\hspace{0.35cm} t = 2,3,\dots , n\\[0.3cm]\]

Observation:

If the avaible data of the process is given by the time series \(\hspace{0.07cm}Y_t = \left( y_1, y_2, ...,y_n \right)^t \hspace{0.1cm}\) , then, for \(\hspace{0.07cm}t=2,...,n\hspace{0.15cm},\) the value \(\hspace{0.15cm}\widehat{y}_t\hspace{0.07cm}\) is an estimation of the observed value \(\hspace{0.08cm}y_t\hspace{0.08cm}\) of the process \(\hspace{0.07cm}\mathcal{Y}\hspace{0.07cm}\).

Note that, an observed value of the process is also a past value of it.

12.2 Forecasting future values of the process

Using an \(\hspace{0.07cm}AR(1)\hspace{0.07cm}\) model we can forcast future values \(\hspace{0.1cm}y_{n+k} \hspace{0.12cm},\hspace{0.12cm} k=1,...,n\hspace{0.1cm}\) of the process using the following equation: \(\\[0.3cm]\)

\[\widehat{y}_{n+1} \hspace{0.1cm}=\hspace{0.1cm} \widehat{\phi}_0 \hspace{0.1cm}+\hspace{0.1cm} \widehat{\phi}_1 \cdot y_{n} \\[0.3cm]\]

\[\widehat{y}_{n+2} \hspace{0.1cm}=\hspace{0.1cm} \widehat{\phi}_0 \hspace{0.1cm}+\hspace{0.1cm} \widehat{\phi}_1 \cdot \widehat{y}_{n+1} \\[0.3cm]\]

\[\widehat{y}_{n+3} \hspace{0.1cm}=\hspace{0.1cm} \widehat{\phi}_0 \hspace{0.1cm}+\hspace{0.1cm} \widehat{\phi}_1 \cdot \widehat{y}_{n+2} \\[0.3cm]\]

In general:

\[ \widehat{y}_{n+k} \hspace{0.1cm}=\hspace{0.1cm} \left\{ \begin{array}{lcc} \widehat{\phi}_0 \hspace{0.1cm}+\hspace{0.1cm} \widehat{\phi}_1 \cdot y_{(n+k)-1} \hspace{0.4cm} , \hspace{0.4cm} \text{if} \hspace{0.3cm} k = 1 \\[0.2cm] \widehat{\phi}_0 \hspace{0.1cm}+\hspace{0.1cm} \widehat{\phi}_1 \cdot \widehat{y}_{(n+k)-1} \hspace{0.4cm} , \hspace{0.4cm} \text{if} \hspace{0.3cm} k = 2,3,\dots \end{array} \right. \\[1.3cm]\]

Note that \(\hspace{0.08cm}(n+k)-1 > n\hspace{0.1cm}\) when \(\hspace{0.08cm}k=2,3,\dots\hspace{0.1cm}\) , and \(\hspace{0.08cm}(n+k)-1 = n\hspace{0.1cm}\) when \(\hspace{0.08cm}k=1\hspace{0.08cm}\).

12.3 Stationariety and forecast

Stationarity is an important property of time series that indicates that the properties of the time series don’t change over time. In other words, the mean, variance, and autocorrelation of a stationary time series are constant over time. \(\hspace{0.08cm}AR(1)\hspace{0.08cm}\) models can be used to model stationary time series.

It is important to note that if a time series isn’t stationary, long-term forecasts may be inaccurate. In this sense, before applying the \(\hspace{0.08cm}AR(1)\hspace{0.08cm}\) model to a time series, it is advisable to verify if it verify the stationarity property. For this, various statistical tests can be performed, such as autocorrelation analysis, unit root test, among others, which will be seen in another specific article.

In the event that a time series is not stationary, transformations can be applied to the data, such as differentiation, that allow obtaining a stationary time series. Then, the \(\hspace{0.08cm}AR(1)\hspace{0.08cm}\) model can be fitted to the differentiated time series and make long-term forecasts. As before, these transformation techniques will be seen in another article with more detail.

In summary, stationarity is an important property of time series for accurate forecasting and \(\hspace{0.08cm}AR(1)\hspace{0.08cm}\) models can be used to model stationary time series. If a time series is not stationary, transformations can be applied to the data to obtain a stationary series and then fit the \(\hspace{0.08cm}AR(1)\hspace{0.08cm}\) model to the transformed time series.

12.4 Estimation of model coeficcients

13.1 Expression of AR(2) as a function of the lag operator

We can express \(\hspace{0.08cm}AR(2)\hspace{0.08cm}\) process using the lag operator as follows: \(\\[0.3cm]\)

\[\left(1 - \phi_1 \cdot B - \phi_2 \cdot B^2 \right) \cdot \mathcal{Y}_t \hspace{0.08cm} =\hspace{0.08cm} \phi_0 \hspace{0.08cm}+\hspace{0.08cm} \varepsilon_t\]

Where:

\(\hspace{0.15cm}B\hspace{0.08cm}\) is the lag operator and is defined as follows:

\[B\cdot \mathcal{Y}_t \hspace{0.1cm}=\hspace{0.1cm} \mathcal{Y}_{t-1}\]

\[B^h\cdot \mathcal{Y}_t \hspace{0.1cm}=\hspace{0.1cm} \mathcal{Y}_{t-h} \hspace{0.3cm} , \hspace{0.3cm} h=1,2,\dots\]

13.2 Mean of AR(2) process

Assuming that \(\hspace{0.07cm}AR(2)\hspace{0.07cm}\) process is weakly stationary in mean we have the following: \(\\[0.4cm]\)

\[\mu_t \hspace{0.07cm}=\hspace{0.07cm} \mu_{\mathcal{Y}} \hspace{0.07cm}=\hspace{0.07cm} \dfrac{\phi_0}{1-\phi_1-\phi_2} \hspace{0.3cm} , \hspace{0.3cm} \forall \hspace{0.07cm} t=1,2,\dots ,n\\\]

Where: \(\hspace{0.12cm}\mu_t \hspace{0.08cm}=\hspace{0.08cm} E[\mathcal{Y}_t]\\\)

Observation:

\(\mu_{\mathcal{Y}}\hspace{0.08cm}\) is finite only if \(\hspace{0.08cm}1-\phi_1-\phi_2 \neq 0\hspace{0.08cm}\).

13.3 Variance of AR(2) process

Assuming that \(\hspace{0.07cm}AR(2)\hspace{0.07cm}\) process is weakly stationary in variance we have the following: \(\\[0.4cm]\)

\[\sigma^2_t \hspace{0.07cm}=\hspace{0.07cm} \sigma_{\mathcal{Y}} \hspace{0.07cm}=\hspace{0.07cm} \dfrac{(1-\phi_2)\cdot \sigma^2}{(1+\phi_2)\cdot (1-\phi_1-\phi_2)\cdot (1+\phi_1 - \phi_2)} \hspace{0.3cm} , \hspace{0.3cm} \forall \hspace{0.07cm} t=1,2,\dots ,n \\\]

Where: \(\hspace{0.12cm}\sigma^2_t \hspace{0.08cm}=\hspace{0.08cm} Var(\mathcal{Y}_t)\\\)

Observation:

\(\sigma^2_{\mathcal{Y}}\hspace{0.11cm}\) is finite and positive only if \(\hspace{0.12cm}\phi_1+\phi_2 < 1\hspace{0.15cm} ,\) \(\hspace{0.15cm}\phi_2-\phi_1 < 1\hspace{0.15cm}\) and \(\hspace{0.15cm}-1< \phi_2 < 1\hspace{0.1cm}\).

13.4 Autocovariance of AR(2) process

Assuming that \(\hspace{0.08cm}AR(1)\hspace{0.08cm}\) process is weakly stationary in covariance, we have the following: \(\\[0.3cm]\)

\[\gamma_{h} \hspace{0.1cm}=\hspace{0.1cm} \phi_1 \cdot \gamma_{h-1} \hspace{0.08cm}+\hspace{0.08cm} \phi_2\cdot \gamma_{h-2} \hspace{0.4cm},\hspace{0.4cm} \forall\hspace{0.08cm} h=0,\pm 1, \pm 2,\dots\]

There isn’t a general expression for \(\hspace{0.08cm}\gamma_{h}\hspace{0.08cm}\) for any \(\hspace{0.08cm}h=0,\pm 1, \pm 2,\dots\hspace{0.08cm}\) , but we can get them by recursion using the previous formula.

\[\gamma_{1} \hspace{0.08cm}=\hspace{0.08cm} \dfrac{\phi_1}{1-\phi_2} \cdot \sigma_{\mathcal{Y}}^2 \\\]

\[\gamma_{2} \hspace{0.08cm}=\hspace{0.08cm} \left( \dfrac{\phi_1^2}{1-\phi_2} + \phi_2 \right)\cdot \sigma_{\mathcal{Y}}^2 \\[1cm]\]

We can prove easily these last results as follow:

\[\gamma_{1} \hspace{0.08cm}=\hspace{0.08cm} \phi_1 \gamma_0 + \phi_2\cdot \gamma_{-1} \hspace{0.08cm}=\hspace{0.08cm} \phi_1 \sigma_{\mathcal{Y}}^2 + \phi_2\cdot \gamma_{1} \hspace{0.3cm}\Rightarrow\hspace{0.3cm} \gamma_{1} \hspace{0.08cm}=\hspace{0.08cm} \dfrac{\phi_1}{1-\phi_2} \cdot \sigma_{\mathcal{Y}}^2\\\]

\[\gamma_{2} \hspace{0.08cm}=\hspace{0.08cm} \phi_1 \gamma_1 + \phi_2\cdot \gamma_{0} \hspace{0.08cm}=\hspace{0.08cm} \phi_1\cdot\dfrac{\phi_1}{1-\phi_2} \cdot \sigma_{\mathcal{Y}}^2 + \phi_2\cdot \sigma_{\mathcal{Y}}^2 \hspace{0.08cm}=\hspace{0.08cm} \left( \dfrac{\phi_1^2}{1-\phi_2} + \phi_2 \right)\cdot \sigma_{\mathcal{Y}}^2\\\]

13.5 Autocorrelation of AR(2)

Assuming that \(\hspace{0.08cm}AR(1)\hspace{0.08cm}\) process is weakly stationary in covariance, we have the following: \(\\[0.3cm]\)

\[\rho_{h} \hspace{0.1cm}=\hspace{0.1cm} \rho_1 \cdot \gamma_{h-1} \hspace{0.08cm}+\hspace{0.08cm} \phi_2\cdot \rho_{h-2} \hspace{0.4cm},\hspace{0.4cm} \forall\hspace{0.08cm} h=0,\pm 1, \pm 2,\dots\]

There isn’t a general expression for \(\hspace{0.08cm}\rho_{h}\hspace{0.08cm}\) for any \(\hspace{0.08cm}h=0,\pm 1, \pm 2,\dots\hspace{0.08cm}\) , but we can get them by recursion using the previous formula.

\[\rho_{1} \hspace{0.08cm}=\hspace{0.08cm} \dfrac{\phi_1}{1-\phi_2} \\\]

\[\rho_{2} \hspace{0.08cm}=\hspace{0.08cm} \dfrac{\phi_1^2}{1-\phi_2} + \phi_2 \\[1cm]\]

We can prove easily these last results considering the relationship between \(\hspace{0.08cm}\rho_{h}\hspace{0.08cm}\) and \(\hspace{0.08cm}\gamma_{h}\hspace{0.08cm}\) that exists in a weakly stationary process in covariance, namely :

\[\rho_{h} \hspace{0.08cm}=\hspace{0.08cm} \dfrac{\gamma_{h}}{\sigma_{\mathcal{Y}}}^2\]

13.6 Weak Stationarity Condition of an AR(2)

The weak stationarity condition is that \(\hspace{0.08cm}| \hspace{0.08cm}G_1\hspace{0.08cm} | < 1\hspace{0.1cm}\) and \(\hspace{0.1cm}| \hspace{0.08cm}G_2\hspace{0.08cm} | < 1\hspace{0.08cm}\).

Where \(\hspace{0.08cm}G_1\hspace{0.08cm}\) and \(\hspace{0.08cm}G_2\hspace{0.08cm}\) are the roots of the following equation, called characteristic equation::

\[1\hspace{0.08cm}-\hspace{0.08cm}\phi_1 \cdot B \hspace{0.08cm}-\hspace{0.08cm} \phi_2 \cdot B^2 \hspace{0.08cm}=\hspace{0.08cm} 0 \\\]

Observation:

The proof that a stationary \(\hspace{0.08cm}AR(2)\hspace{0.08cm}\) process requires the roots of the characteristic equation to be inside the unit circle is somewhat involved and relies on some results from linear algebra and complex analysis. So, in order to not make more complex this article, we are not going to show the proof here.

14.1 Estimating past values of the process

Using an \(\hspace{0.08cm}AR(2)\hspace{0.08cm}\) model, the estimation of the past (observed) values \(\hspace{0.1cm}y_t \hspace{0.1cm},\hspace{0.1cm} t=1,...,n\) of the process is done using the following equation: \(\\[0.3cm]\)

\[\widehat{y}_t \hspace{0.1cm}=\hspace{0.1cm} \widehat{\phi}_0 \hspace{0.1cm}+\hspace{0.1cm} \widehat{\phi}_1 \cdot y_{t-1} \hspace{0.1cm}+\hspace{0.1cm} \widehat{\phi}_2 \cdot y_{t-2} \hspace{0.35cm},\hspace{0.35cm} t = 3,4,\dots , n\\[0.3cm]\]

Observation:

If the avaible data of the process is given by the time series \(\hspace{0.07cm}Y_t = \left( y_1, y_2, ...,y_n \right)^t \hspace{0.1cm}\) , then, for \(\hspace{0.07cm}t=3,4,...,n\hspace{0.15cm},\) the value \(\hspace{0.15cm}\widehat{y}_t\hspace{0.07cm}\) is an estimation of the observed value \(\hspace{0.08cm}y_t\hspace{0.08cm}\) of the process \(\hspace{0.07cm}\mathcal{Y}\hspace{0.07cm}\).

Note that, an observed value of the process is also a past value of it.

14.2 Forecasting future values of the process

Using an \(\hspace{0.07cm}AR(2)\hspace{0.07cm}\) model we can forecast future values \(\hspace{0.1cm}y_{n+k} \hspace{0.12cm},\hspace{0.12cm} k=1,...,n\hspace{0.1cm}\) of the process using the following equation: \(\\[0.3cm]\)

\[\widehat{y}_{n+1} \hspace{0.1cm}=\hspace{0.1cm} \widehat{\phi}_0 \hspace{0.1cm}+\hspace{0.1cm} \widehat{\phi}_1 \cdot y_{n} \hspace{0.1cm}+\hspace{0.1cm} \widehat{\phi}_2 \cdot y_{n-1} \\[0.3cm]\]

\[\widehat{y}_{n+2} \hspace{0.1cm}=\hspace{0.1cm} \widehat{\phi}_0 \hspace{0.1cm}+\hspace{0.1cm} \widehat{\phi}_1 \cdot \widehat{y}_{n+1} \hspace{0.1cm}+\hspace{0.1cm} \widehat{\phi}_2 \cdot y_{n} \\[0.3cm]\]

\[\widehat{y}_{n+3} \hspace{0.1cm}=\hspace{0.1cm} \widehat{\phi}_0 \hspace{0.1cm}+\hspace{0.1cm} \widehat{\phi}_1 \cdot \widehat{y}_{n+2} \hspace{0.1cm}+\hspace{0.1cm} \widehat{\phi}_2 \cdot \widehat{y}_{n+1} \\[0.35cm]\]

In general:

\[ \widehat{y}_{n+k} \hspace{0.1cm}=\hspace{0.1cm} \left\{ \begin{array}{lcc} \widehat{\phi}_0 \hspace{0.1cm}+\hspace{0.1cm} \widehat{\phi}_1 \cdot y_{(n+k)-1} \hspace{0.1cm}+\hspace{0.1cm} \widehat{\phi}_2 \cdot y_{(n+k)-2} \hspace{0.4cm} , \hspace{0.4cm} \text{if} \hspace{0.3cm} k = 1 \\[0.2cm] \widehat{\phi}_0 \hspace{0.1cm}+\hspace{0.1cm} \widehat{\phi}_1 \cdot \hat{y}_{(n+k)-1} \hspace{0.1cm}+\hspace{0.1cm} \widehat{\phi}_2 \cdot y_{(n+k)-2} \hspace{0.4cm} , \hspace{0.4cm} \text{if} \hspace{0.3cm} k = 2 \\[0.2cm] \widehat{\phi}_0 \hspace{0.1cm}+\hspace{0.1cm} \widehat{\phi}_1 \cdot \widehat{y}_{(n+k)-1} \hspace{0.1cm}+\hspace{0.1cm} \widehat{\phi}_2 \cdot \widehat{y}_{(n+k)-2}\hspace{0.4cm} , \hspace{0.4cm} \text{if} \hspace{0.3cm} k = 3,4,\dots \end{array} \right. \\[1.3cm]\]

14.3 Stationariety and forecast

Stationarity is an important property of time series that indicates that the properties of the time series don’t change over time. In other words, the mean, variance, and autocorrelation of a stationary time series are constant over time. \(\hspace{0.08cm}AR(2)\hspace{0.08cm}\) models can be used to model stationary time series.

It is important to note that if a time series isn’t stationary, long-term forecasts may be inaccurate. In this sense, before applying the \(\hspace{0.08cm}AR(2)\hspace{0.08cm}\) model to a time series, it is advisable to verify if it verify the stationarity property. For this, various statistical tests can be performed, such as autocorrelation analysis, unit root test, among others, which will be seen in another specific article.

In the event that a time series is not stationary, transformations can be applied to the data, such as differentiation, that allow obtaining a stationary time series. Then, the \(\hspace{0.08cm}AR(2)\hspace{0.08cm}\) model can be fitted to the differentiated time series and make long-term forecasts. As before, these transformation techniques will be seen in another article with more detail.

In summary, stationarity is an important property of time series for accurate forecasting and \(\hspace{0.08cm}AR(2)\hspace{0.08cm}\) models can be used to model stationary time series. If a time series is not stationary, transformations can be applied to the data to obtain a stationary series and then fit the \(\hspace{0.08cm}AR(2)\hspace{0.08cm}\) model to the transformed time series.

14.4 Estimation of model coeficcients

15.1 Matrix expression of AR(p)

Taking into account the following matrix:

\[ \mathbf{Y} = \begin{pmatrix} \mathcal{Y}_{p+1} \\ \mathcal{Y}_{p+2}\\ ... \\ \mathcal{Y}_{n} \end{pmatrix} \]

\[ \mathbf{X} = \begin{pmatrix} 1 & \mathcal{Y}_p & \mathcal{Y}_{p-1} & ... & \mathcal{Y}_{1} \\ 1 & \mathcal{Y}_{p+1} & \mathcal{Y}_{p} & ... & \mathcal{Y}_{2} \\ ... \\ 1 & \mathcal{Y}_{n-1} & \mathcal{Y}_{n-2} & ... & \mathcal{Y}_{n-p} \\ \end{pmatrix} \]

\[ \boldsymbol{\phi} = \begin{pmatrix} \phi_0 \\ \phi_1 \\ ... \\ \phi_p \end{pmatrix} \]

\[ \boldsymbol{\varepsilon} = \begin{pmatrix} \varepsilon_{p+1} \\ \varepsilon_{p+2} \\ ... \\ \varepsilon_{n} \end{pmatrix} \]

We can express an AR(p) process as follow:

\[\mathbf{Y} \hspace{0.08cm}=\hspace{0.08cm} \mathbf{X} \cdot \boldsymbol{\phi} \hspace{0.08cm}+\hspace{0.08cm} \boldsymbol{\varepsilon}\]

15.2 Expression of AR(p) as a function of the lag operator

We can express \(\hspace{0.08cm}AR(p)\hspace{0.08cm}\) process using the lag operator as follows: \(\\[0.3cm]\)

\[\phi_p(B)\cdot \mathcal{Y}_t \hspace{0.08cm} =\hspace{0.08cm}\left(1 - \phi_1 \cdot B - \phi_2 \cdot B^2 - \dots - \phi_p \cdot B^p \right) \cdot \mathcal{Y}_t \hspace{0.08cm} =\hspace{0.08cm} \phi_0 \hspace{0.08cm}+\hspace{0.08cm} \varepsilon_t\]

Where:

\(\hspace{0.15cm}B\hspace{0.08cm}\) is the lag operator and is defined as follows:

\[B\cdot \mathcal{Y}_t \hspace{0.1cm}=\hspace{0.1cm} \mathcal{Y}_{t-1}\]

\[B^h\cdot \mathcal{Y}_t \hspace{0.1cm}=\hspace{0.1cm} \mathcal{Y}_{t-h} \hspace{0.3cm} , \hspace{0.3cm} h=1,2,\dots\]

15.3 Mean of AR(p) process

Assuming that \(\hspace{0.07cm}AR(p)\hspace{0.07cm}\) process is weakly stationary in mean we have the following: \(\\[0.4cm]\)

\[\mu_t \hspace{0.07cm}=\hspace{0.07cm} \mu_{\mathcal{Y}} \hspace{0.07cm}=\hspace{0.07cm} \dfrac{\phi_0}{1-\phi_1-\phi_2-\dots -\phi_p}\hspace{0.07cm}=\hspace{0.07cm} \dfrac{\phi_0}{1- \sum_{i=1}^p \phi_i} \hspace{0.35cm} , \hspace{0.35cm} \forall \hspace{0.07cm} t=1,2,\dots ,n\\\]

Where: \(\hspace{0.12cm}\mu_t \hspace{0.08cm}=\hspace{0.08cm} E[\mathcal{Y}_t]\\\)

Observation:

\(\mu_{\mathcal{Y}}\hspace{0.1cm}\) is finite only if \(\hspace{0.1cm}1-\phi_1-\phi_2- \dots - \phi_p \neq 0\hspace{0.08cm}\).

15.4 Variance of AR(p) process

Assuming that \(\hspace{0.07cm}AR(p)\hspace{0.07cm}\) process is weakly stationary in variance we have the following: \(\\[0.4cm]\)

\[\sigma_t^2 \hspace{0.08cm}=\hspace{0.08cm} \sigma_{\mathcal{Y}}^2 \hspace{0.08cm}=\hspace{0.08cm} \sigma^2 \hspace{0.08cm}+\hspace{0.08cm} \sum_{h=1}^p\hspace{0.08cm} \phi_h \gamma_{h}\]

Where:

\(\hspace{0.12cm}\sigma^2_t \hspace{0.08cm}=\hspace{0.08cm} Var(\mathcal{Y}_t)\\\)

\(\gamma_{h} = Cov(\mathcal{Y}_{t}, \mathcal{Y}_{t-h}) , t =1,\dots , n\)

15.5 Autocovariance of AR(p) process

There isn’t a general expression for \(\hspace{0.08cm}\gamma_{h}\hspace{0.08cm}\) for any \(\hspace{0.08cm}h=0,\pm 1, \pm 2,\dots\hspace{0.08cm}\).

But, assuming that \(\hspace{0.08cm}AR(p)\hspace{0.08cm}\) process is weakly stationary in covariance, we have a usuful result, the Yule-Walker equations. \(\\[0.3cm]\)

The Yule-Walker equations for an \(\hspace{0.08cm}AR(p)\hspace{0.08cm}\) process are a system of \(\hspace{0.08cm}p\hspace{0.08cm}\) linear equations that relate the autocovariances at lags \(\hspace{0.08cm}1, 2, ..., p\hspace{0.08cm}\) to the coefficients \(\hspace{0.08cm}\phi_1, \phi_2, \ldots, \phi_p\hspace{0.08cm}\).

These equations are:

\[ \gamma_1 \hspace{0.08cm}=\hspace{0.08cm} \phi_1\cdot \gamma_0 \hspace{0.08cm}+\hspace{0.08cm} \phi_2\cdot \gamma_1 \hspace{0.08cm}+ \cdots +\hspace{0.08cm} \phi_p \cdot\gamma_{p-1} \\ \gamma_2 \hspace{0.08cm}=\hspace{0.08cm} \phi_1 \cdot\gamma_1 \hspace{0.08cm}+\hspace{0.08cm} \phi_2 \cdot\gamma_0 \hspace{0.08cm}+ \cdots +\hspace{0.08cm} \phi_p \cdot\gamma_{p-2} \\ \vdots \\ \gamma_p \hspace{0.08cm}=\hspace{0.08cm} \phi_1\cdot \gamma_{p-1} \hspace{0.08cm}+\hspace{0.08cm} \phi_2\cdot \gamma_{p-2} \hspace{0.08cm}+ \cdots +\hspace{0.08cm} \phi_{p}\cdot \gamma_0 \\\]

These equations can be solved to obtain estimates of the coefficients \(\hspace{0.08cm}\phi_1, \phi_2, \ldots, \phi_p\hspace{0.08cm}\) based on the sample estimation of autocovariances \(\hspace{0.08cm}\widehat{\gamma}_1, \widehat{\gamma}_2, \ldots, \widehat{\gamma}_p\hspace{0.08cm}\). The solution involves inverting a system of linear equations, which can be done numerically.

15.6 Autocorrelation of AR(p)

There isn’t a general expression for \(\hspace{0.08cm}\rho_{h}\hspace{0.08cm}\) for any \(\hspace{0.08cm}h=0,\pm 1, \pm 2,\dots\hspace{0.08cm}\).

But, assuming that \(\hspace{0.08cm}AR(p)\hspace{0.08cm}\) process is weakly stationary in covariance, we have a usuful result, the Yule-Walker equations. \(\\[0.3cm]\)

The Yule-Walker equations for an \(\hspace{0.08cm}AR(p)\hspace{0.08cm}\) process are a system of \(\hspace{0.08cm}p\hspace{0.08cm}\) linear equations that relate the autocorrelation at lags \(\hspace{0.08cm}1, 2, ..., p\hspace{0.08cm}\) to the coefficients \(\hspace{0.08cm}\phi_1, \phi_2, \ldots, \phi_p\hspace{0.08cm}\).

These equations are:

\[ \rho_1 \hspace{0.08cm}=\hspace{0.08cm} \phi_1 \hspace{0.08cm}+\hspace{0.08cm} \phi_2\cdot \rho_1 \hspace{0.08cm}+ \cdots +\hspace{0.08cm} \phi_p \cdot\rho_{p-1} \\ \rho_2 \hspace{0.08cm}=\hspace{0.08cm} \phi_1 \cdot\rho_1 \hspace{0.08cm}+\hspace{0.08cm} \phi_2 \hspace{0.08cm}+ \cdots +\hspace{0.08cm} \phi_p \cdot\rho_{p-2} \\ \vdots \\ \rho_p \hspace{0.08cm}=\hspace{0.08cm} \phi_1\cdot \rho_{p-1} \hspace{0.08cm}+\hspace{0.08cm} \phi_2\cdot \rho_{p-2} \hspace{0.08cm}+ \cdots +\hspace{0.08cm} \phi_{p}\cdot \gamma_0 \\ \]

These equations can be solved to obtain estimates of the coefficients \(\hspace{0.08cm}\phi_1, \phi_2, \ldots, \phi_p\hspace{0.08cm}\) based on the sample estimation of autocorrelations \(\hspace{0.08cm}\widehat{\rho}_1, \widehat{\rho}_2, \ldots, \widehat{\rho}_p\hspace{0.08cm}\). The solution involves inverting a system of linear equations, which can be done numerically.

15.7 Weak Stationarity Condition of an AR(p)

The weak stationarity condition is that \(\hspace{0.08cm}| \hspace{0.08cm}G_i\hspace{0.08cm} | < 1\hspace{0.2cm} ,\hspace{0.2cm} \forall i =1,\dots ,p\hspace{0.08cm}\).

Where \(\hspace{0.08cm}G_i\hspace{0.08cm}\hspace{0.2cm} ,\hspace{0.2cm} \forall i =1,\dots ,p\hspace{0.08cm}\) are the roots of the following equation, called characteristic equation: \(\\[0.35cm]\)

\[\phi_p(B) \hspace{0.08cm}=\hspace{0.08cm} 1\hspace{0.08cm}-\hspace{0.08cm}\phi_1 \cdot B \hspace{0.08cm}-\hspace{0.08cm} \phi_2 \cdot B^2 \hspace{0.08cm}-\hspace{0.08cm} \dots \hspace{0.08cm}-\hspace{0.08cm} \phi_p \cdot B^p \hspace{0.08cm}=\hspace{0.08cm} 0 \\\]

Observation:

The proof that a stationary \(\hspace{0.08cm}AR(p)\hspace{0.08cm}\) process requires the roots of the characteristic equation to be inside the unit circle is somewhat involved and relies on some results from linear algebra and complex analysis. So, in order to not make more complex this article, we are not going to show the proof here.

16.1 Estimating past values of the process

Using an \(\hspace{0.08cm}AR(p)\hspace{0.08cm}\) model, the estimation of the past (observed) values \(\hspace{0.1cm}y_t \hspace{0.1cm},\hspace{0.1cm} t=1,...,n\) of the process is done using the following equation: \(\\[0.3cm]\)

\[\widehat{y}_t \hspace{0.1cm}=\hspace{0.1cm} \widehat{\phi}_0 \hspace{0.1cm}+\hspace{0.1cm} \widehat{\phi}_1 \cdot y_{t-1} \hspace{0.1cm}+\hspace{0.1cm} \widehat{\phi}_2 \cdot y_{t-2} \hspace{0.1cm}+\hspace{0.1cm} \dots \hspace{0.1cm}+\hspace{0.1cm} \widehat{\phi}_p \cdot y_{t-p} \hspace{0.1cm}=\hspace{0.1cm} \phi_0 \hspace{0.1cm}+\hspace{0.1cm} \sum_{h=1}^p \widehat{\phi}_h \cdot y_{t-h} \hspace{0.35cm},\hspace{0.35cm} t = p+1,\dots , n\\[0.4cm]\]

Observation:

If the avaible data of the process is given by the time series \(\hspace{0.07cm}Y_t = \left( y_1, y_2, ...,y_n \right)^t \hspace{0.12cm}\) , then, for \(\hspace{0.07cm}t=p+1,...,n,\hspace{0.15cm}\) the value \(\hspace{0.15cm}\widehat{y}_t\hspace{0.07cm}\) is an estimation of the observed value \(\hspace{0.08cm}y_t\hspace{0.08cm}\) of the process \(\hspace{0.07cm}\mathcal{Y}\hspace{0.07cm}\).

Note that, an observed value of the process is also a past value of it.

16.2 Forecasting future values of the process

Using an \(\hspace{0.07cm}AR(p)\hspace{0.07cm}\) model we can forecast future values \(\hspace{0.1cm}y_{n+k} \hspace{0.12cm},\hspace{0.12cm} k=1,...,n\hspace{0.12cm}\) of the process using the following equations: \(\\[0.3cm]\)

\[\widehat{y}_{n+1} \hspace{0.1cm}=\hspace{0.1cm} \phi_0 \hspace{0.05cm}+\hspace{0.05cm} \sum_{h=1}^p \hspace{0.05cm}\widehat{\phi}_h \cdot y_{(n+1) - h} \hspace{0.1cm}=\hspace{0.1cm} \phi_0 \hspace{0.07cm}+\hspace{0.07cm} \widehat{\phi}_1 \cdot y_{n} \hspace{0.07cm}+\hspace{0.07cm} \widehat{\phi}_2\cdot y_{n-1} \hspace{0.07cm}+\hspace{0.07cm} \cdots \hspace{0.07cm}+\hspace{0.07cm} \widehat{\phi}_p \cdot y_{n-(p-1)} \\[0.25cm]\]

\[\widehat{y}_{n+2} \hspace{0.1cm}=\hspace{0.1cm} \phi_0 \hspace{0.07cm}+\hspace{0.07cm} \widehat{\phi}_1 \cdot \widehat{y}_{(n+2)-1} \hspace{0.07cm}+\hspace{0.07cm} \sum_{h=2}^p \hspace{0.05cm} \widehat{\phi}_h \cdot y_{(n+2) - h} \\[0.3cm]\]

\[\widehat{y}_{n+3} \hspace{0.1cm}=\hspace{0.1cm} \phi_0 \hspace{0.07cm}+\hspace{0.07cm} \widehat{\phi}_1 \cdot \widehat{y}_{(n+3)-1} \hspace{0.07cm}+\hspace{0.07cm} \widehat{\phi}_2 \cdot \widehat{y}_{(n+3)-2} \hspace{0.07cm}+\hspace{0.07cm} \sum_{h=3}^p \hspace{0.05cm} \widehat{\phi}_h \cdot y_{(n+3) - h} \\[1.5cm]\]

• In general, for \(\hspace{0.1cm} k=1,2,3,\dots\hspace{0.1cm}\) , we have the following prediction equation: \(\\[0.35cm]\)

  \[\widehat{y}_{n+k} \hspace{0.13cm}=\hspace{0.13cm} \phi_0 \hspace{0.1cm}+\hspace{0.1cm} \sum_{h=1}^{k-1} \hspace{0.08cm} \widehat{\phi}_h \cdot \widehat{y}_{(n+k)-h} \hspace{0.1cm}+\hspace{0.1cm} \sum_{h=k}^p \hspace{0.08cm} \widehat{\phi}_h \cdot y_{(n+k) - h} \hspace{0.13cm}=\hspace{0.13cm} \phi_0 \hspace{0.1cm}+\hspace{0.1cm} \widehat{\phi}_1 \cdot \widehat{y}_{(n+k)-1} \hspace{0.1cm}+\hspace{0.1cm} \widehat{\phi}_2 \cdot \widehat{y}_{(n+k)-2} \hspace{0.1cm}+\hspace{0.1cm} \dots\hspace{0.1cm}+\hspace{0.1cm} \widehat{\phi}_{k-1} \cdot \widehat{y}_{(n+k)-(k-1)} \hspace{0.05cm}+\hspace{0.05cm}\widehat{\phi}_k \cdot \widehat{y}_{(n+k)-k } \hspace{0.05cm}+\hspace{0.05cm} \widehat{\phi}_{k+1}\cdot y_{(n+k)-(k+1) }\hspace{0.05cm}+\hspace{0.05cm}\widehat{\phi}_{k+2}\cdot y_{(n+k)-(k+2)} \hspace{0.05cm}+\hspace{0.05cm}\cdots \hspace{0.05cm}+\hspace{0.05cm} \widehat{\phi}_{p} \cdot y_{(n+k)-p} \\[0.45cm]\]

  Where, it must be assumed that \(\hspace{0.09cm}n+k \hspace{0.05cm}> \hspace{0.05cm}p\hspace{0.05cm}\).

16.3 Stationariety and forecast

Stationarity is an important property of time series that indicates that the properties of the time series don’t change over time. In other words, the mean, variance, and autocorrelation of a stationary time series are constant over time. \(\hspace{0.08cm}AR(p)\hspace{0.08cm}\) models can be used to model stationary time series.

It is important to note that if a time series isn’t stationary, long-term forecasts may be inaccurate. In this sense, before applying the \(\hspace{0.08cm}AR(p)\hspace{0.08cm}\) model to a time series, it is advisable to verify if it verify the stationarity property. For this, various statistical tests can be performed, such as autocorrelation analysis, unit root test, among others, which will be seen in another specific article.

In the event that a time series is not stationary, transformations can be applied to the data, such as differentiation, that allow obtaining a stationary time series. Then, the \(\hspace{0.08cm}AR(p)\hspace{0.08cm}\) model can be fitted to the differentiated time series and make long-term forecasts. As before, these transformation techniques will be seen in another article with more detail.

In summary, stationarity is an important property of time series for accurate forecasting and \(\hspace{0.08cm}AR(p)\hspace{0.08cm}\) models can be used to model stationary time series. If a time series is not stationary, transformations can be applied to the data to obtain a stationary series and then fit the \(\hspace{0.08cm}AR(p)\hspace{0.08cm}\) model to the transformed time series.

16.4 Estimation of model coeficcients